1. Field of the Invention
The present invention relates generally to the field of virtual reality simulation, and more particularly to systems for combining virtual images with real-world scenes.
2. Statement of the Problem
Virtual reality systems have gained wide acceptance in recent years for training and simulations in a number of fields, including aircraft simulators, virtual reality games, surgical training, military training and motion picture and video special effects. Conventional virtual reality systems generate a field of view for the user that is either completely computer-generated, or may include real-world scenery as background. Some virtual reality systems use portions of real-world images (e.g., a particular object, pattern, or texture) that can be incorporated into a computer-generated environment. However, conventional virtual reality systems do not typically incorporate virtual images into real-world scenes due to the difficulties of integrating a virtual image into a real-world scene in a realistic manner. For example, the virtual image can mask portions of the real-world scene, and objects in the real-world scene can mask portions of the virtual image depending on their relative locations and sizes.
Combining virtual images with real-world scenes will improve many current applications of virtual reality, and it will make possible new applications and systems for applying virtual reality. For example, the U.S. Army trains its armored vehicle crews using virtual reality simulators. These simulators are static and rely entirely on computer-generated images. The army also trains armored vehicle crews in war games using actual vehicles moving over real terrain. These vehicles can be fully instrumented and connected to computer systems that record and analyze the performance of the crew and vehicle throughout the exercise. The computer system can also record and indicate simulated "hits" on opposing vehicles. Although such war games are much more realistic than simulators, they are also much more expensive and time consuming due to the requirements of operating actual vehicles and managing the manpower necessary to conduct a realistic exercise. The cost of instrumentation for a large number of vehicles is also substantial. The present invention can be used in these war games to generate virtual tanks that appear to move about the terrain. This can completely eliminate the need, for example, for live tanks representing the enemy force in a war gaming exercise. Virtual vehicles and real vehicles can appear in the same scene. The present invention can also be used to generate virtual "hits" and explosions during an exercise.
The present invention also has application in the field of aircraft simulation. Virtual aircraft or virtual flying conditions can be combined with real-world scenes during an actual flight or a simulated flight. Another possible field of use is in games, such as laser tag, so that virtual players, objects, and special effects can be combined with real players and real objects in the playing field.
The present invention has application in the area of entertainment in the high-speed generation of special effects and three-dimensional displays. Special effects in the motion picture and video industries integrate real-world scenes and computer graphics, that is, virtual images. The invention permits real-time generation of special effects in video film and motion picture production. Also, there exist various three-dimensional motion picture and video systems that can replay, with the aid of special display goggles, recorded three-dimensional scenes. The present invention can be used with electronic display goggles to greatly extend this entertainment medium by injecting, in real time, three-dimensional virtual elements into the view of the real world. The method of the present invention can be also used to create virtual elements in theme parks and other entertainment-oriented environments. The virtual elements injected into real-world environments in entertainment applications may be representations of persons, animals, natural phenomena, cartoon characters, and other real or imaginary visual elements.
3. Solution to the Problem
The present system enables computer-generated virtual images to be combined with images of the real world. A range scanner determines the shape and distance of real-world objects within a field of regard of interest to the observer. Virtual masking objects, which are simplified computer models of real-world objects, are derived from the range data. A computer simulates virtual entities and combines these virtual images with the virtual masking objects to create masked virtual images. The masked virtual images show the portions of virtual entities that would be visible if the virtual entities actually existed in the real world. The masked virtual images and the real-world scene are combined and displayed in such a manner that the virtual images appear to be obscured, when appropriate for their simulated location relative to real-world objects.